Search for: All records

Physical computing enables learners to create interactive projects using tangible materials and electronic components. These projects commonly utilize microcontroller boards like the micro:bit. In contrast, computer vision (CV) is a powerful technique for detecting input through interaction with everyday materials like paper, and it can be utilized for physical computing projects. However, CV-based toolkits are typically limited to input detection and rely on screen-based or projected outputs. This paper presents a hybrid approach that integrates a CV-based platform called Paper Playground with the micro:bit electronics platform. By combining CV-detected, paper-based inputs with the rich input-output possibilities of microcontroller-based systems, we showcase a multimodal physical computing toolkit. Through three project examples, we explore how this hybrid approach can enhance the creative possibilities in physical computing, and develop a preliminary design space combining CV-based and electronics-based physical computing. 

In-person human interaction relies on our spatial perception of each other and our surroundings. Current remote communication tools partially address each of these aspects. Video calls convey real user representations but without spatial interactions. Augmented and Virtual Reality (AR/VR) experiences are immersive and spatial but often use virtual environments and characters instead of real-life representations. Bridging these gaps, we introduce DualStream, a system for synchronous mobile AR remote communication that captures, streams, and displays spatial representations of users and their surroundings. DualStream supports transitions between user and environment representations with different levels of visuospatial fidelity, as well as the creation of persistent shared spaces using environment snapshots. We demonstrate how DualStream can enable spatial communication in real-world contexts, and support the creation of blended spaces for collaboration. A formative evaluation of DualStream revealed that users valued the ability to interact spatially and move between representations, and could see DualStream fitting into their own remote communication practices in the near future. Drawing from these findings, we discuss new opportunities for designing more widely accessible spatial communication tools, centered around the mobile phone. 

We present Beholder, a computer vision (CV) toolkit for building tangible controllers for interactive computer systems. Beholder facilitates designers to build physical inputs that are instrumented with CV markers. By observing the properties of these markers, a CV system can detect physical interactions that occur. Beholder provides a software editor that enables designers to map CV marker behavior to keyboard events; thus connecting the CV-driven tangible controllers to any software that responds to keyboard input. We propose three design scenarios for Beholder—controllers to support everyday work, alternative controllers for games, and transforming physical therapy equipment into controllers to monitor patient progress. 

The emergence of on-skin interfaces has created an opportunity for seamless, always-available on-body interactions. However, developing a new fabrication process for on-skin interfaces can be time-consuming, challenging to incorporate new features, and not available for quick form-factor preview through prototyping. We introduce SkinKit, the first construction toolkit for on-skin interfaces, which enables fast, low-fidelity prototyping with a slim form factor directly applicable to the skin. SkinKit comprises modules consisting of skin-conformable base substrates and reusable Flexible Printed Circuits Board (FPCB) blocks. They are easy to attach and remove under tangible plug-and-play construction but still offer robust conductive connections in a slim form. Further, SkinKit aims to lower the barrier to entry in building on-skin interfaces without demanding technical expertise. It leverages a variety of preprogrammed modules connected in unique sequences to achieve various function customizations. We describe our iterative design and development process of SkinKit, comparing materials, connection mechanisms, and modules reflecting on its capability. We report results from single- and multi- session workshops with 34 maker participants spanning STEM and design backgrounds. Our findings reveal how diverse maker populations engage in on-skin interface design, what types of applications they choose to build, and what challenges they faced.